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Abstract 

Virtualization  is  a  new  revolutionary  approach  in  networking  industry,  its  make  possible  to  build 
several  virtual  machine  (VM)  in  one  physical  hardware.  In  virtualization  practice,  one  VM  might  be 
connected  to  others,  but  not  all  of  VM  in  one  environment  must  be  connected  due  the  privacy  and  security 
issues.  One  of  the  solutions  which  can  address  this  issue  is  tunneling  protocol.  Tunneling  protocol  is  a 
layer-2-in-layer-3  protocol  which  can  isolate  tenant  traffic  in  virtualize  environment.  This  research 
conducted  about  the  performance  of  VXLAN  and  NVGRE  tunneling  protocol  which  works  on  virtualize 
environment  and  aims  to  determine  the  perfomances  of  throughput,  delay,  jitter,  and  vCPU  Usage  using 
variable  packet  size  in  range  of  128-1514  byte.  From  the  the  result,  can  be  conclude  that  both  of  tunneling 
protocol  can  isolate  the  traffic  between  tenant.  For  the  performance  result,  NVGRE  has  the  highest  value 
of  throughput,  771,02  Mbps  and  the  VXLAN  got  753,62  Mbps.  For  the  delay  NVGRE  got  2.24  ms  and 
VXLAN  got  2.29  ms.  For  the  jitter,  NVGRE  has  better  rate  value  of  0.361  ms,  than  VXLAN  value  of  0.348 
ms,  and  the  vCPU  usage  performance,  NVGRE  has  the  highest  performance  too  that  value  is  60.57%.  So 
on  overall  performance  NVGRE  has  the  better  performance  than  VXLAN. 
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1.  Introduction 

In  this  era  of  global  competition,  the  availability  of  cloud  computing  infrastructure  in  a 
corporate  or  agency  environment  becomes  very  important.  In  practice,  there  are  often  occurs 
problems  faced.  Those  problems  are  the  scalability  and  flexibility  of  the  cloud  computing 
infrastructure  [1].  As  the  massive  ICT  development,  one  the  emerging  solution  to  simplify  these 
problems  is  approaching  virtualization  technology.  Which  has  been  widely  implemented  by 
small  and  large  scale  ICT  companies  like  datacenter  [1],  This  virtualization  technology  can’t  be 
denied  that  can  decreasing  the  CAPEX  and  OPEX  side  [2], 

Virtualization  is  the  use  of  software  to  allocate  the  resources  of  hardware  into  several 
parts  like  virtual  machine  (VM).  With  virtualization,  the  network  complexity  can  be  simplified,  and 
can  be  deployed  according  to  user  needs  flexibly  and  scalable  [3].  In  datacenter-scale 
virtualization,  which  can  accommodate  a  large  number  of  tenants.  Each  tenant  may  be  has 
more  than  one  integrated  VM  to  execute  an  application,  so  each  tenant  must  be  isolated  from 
others  [4],  One  of  the  solution  that  arises  to  address  those  problems  is  a  tunneling  protocol 
technology.  This  protocol  is  able  to  isolate  traffic  within  the  virtualization  technology  which  called 
overlay  virtual  network  (OVN)  [5].  There  are  also  another  way  that  use  of  VLAN  technology 
(802.1  Q).  However,  this  technology  is  considered  that  not  reliable  enough  because  of  scalability 
issue.  VLAN  relatively  low  if  its  compare  with  the  tunneling  protocol  technology,  because  VLAN 
only  has  the  capacity  of  VLAN  tag  around  4000  VLAN  ID. 

Tunneling  protocols  that  works  in  virtual  environments  or  called  with  overlay  virtual 
network,  has  three  protocols,  namely  VXLAN,  NVGRE,  and  STT  [6].  These  tunneling  protocols 
can  isolate  tenant  traffic  up  to  16  million  tunnel  ids  [6],  coupled  with  a  multitenancy  features  that 
increase  tenant  flexibility.  But  in  implementation  of  this  technology  there  are  some  shortcoming 
issues,  which  can  burden  of  vCPU  usage  and  bandwidth  effectiveness,  these  because  the 
addition  of  encapsulation  process  to  the  packet  before  its  transmitted,  so  the  vCPU  usage 
needs  to  work  more  than  before,  along  with  the  increasing  the  packet  size  as  well  [6].  At  this 
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research  will  analyze  the  performance  comparison  of  the  two  tunneling  protocols,  there  are 
NVGRE  and  VXLAN.  It  aims  to  see  which  one  has  the  better  performance. 


2.  Research  Method 

This  research  used  system  architecture  as  shown  in  Figure  1.  and  the  topology  shown 
in  Figure  2.  which  consists  of  two  servers,  user  PC  and  one  switch.  Each  server  is  built  in  a 
virtual  environment  using  the  Xen  Server  hypervisor  and  thereafter  built  one  VM  from  Ubuntu 
Linux  as  host.  User  PC  is  used  as  hypervisor  manager  by  installing  Xen  Center  which  is  the 
default  application  of  Xen  Hypervisor  that  serves  to  control  and  monitoring  virtual  resources 
remotely.  Once  the  system  is  ready,  the  tunneling  protocol  is  built  using  bash  scripting  on  each 
VM  that  has  previously  been  installed  Open  vSwitch.  Open  vSwitch  is  use  as  a  tunnel  end  point 
to  implementing  this  tunneling  protocol  at  the  system  [7], 


Figure  1.  Tunneling  system  architecture 


There  are  four  test  scenarios  performed  to  see  the  performance  of  the  tunneling 
protocol  VXLAN  and  NVGRE,  which  work  in  virtual  environment  [8].  In  addition,  the  two  of 
tunneling  protocols  were  also  tested  native-based  communication  (non-tunneling),  as  a 
comparison  with  the  two  tunneling  protocols  tested.  There  are  4  parameters  that  tested  in  these 
experiments,  those  are  throughput  testing,  delay,  jitter,  and  vCPU  Usage  [9].  This  test  is  based 
on  the  size  of  the  packet  size  transmitted  by  VM  A  as  the  sender,  to  the  VM  B  as  the  receiver, 
which  is  64,  128,  256,  384,  512,  640,  768,  896,  1024,  1153,  1280,  1408,  and  1514  bytes  [9]. 

These  metric  value  were  measured  by  using  D-ITG  application.  The  D-ITG  has  capable 
to  producing  traffic  with  various  packet  level.  D-ITG  can  be  used  to  measure  throughput,  delay, 
jitter  across  these  virtual  environment  system  [9-11].  To  ensure  the  data  accuracy,  all  the  test 
were  execute  20  times,  and  each  run  had  duration  of  10  second  [12-13].  Table  1  show  virtual 
machine  specification 


Table  1.  Virtual  Machine  Specification 
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Figure  2.  Tunneling  system  architecture 


3.  Results  and  Analysis 

The  test  is  conducted  to  find  out  which  tunneling  protocol  has  the  most  optimal 
throughput  value  and  measure  how  far  the  trade-off  occurs  between  VXLAN  and  NVGRE  and 
also  with  native.  Based  on  Figure  3,  it  can  be  seen  that  there  is  difference  of  throughput  value 
between  VXLAN  and  NVGRE  performance.  The  difference  occurs  when  the  size  of  the  tested 
packet  size  is  128  -1514  bytes.  NVGRE  performance  has  a  greater  throughput  value  than 
VXLAN  performance  throughout  the  packet  size  testing  is  increased. 


Throughput 


♦  Native 
—•—NVGRE 
—A— VXLAN 


Figure  3.  Throughput  performance  from  VM-A  to  VM-B 


The  test  is  conducted  to  find  out  which  tunneling  protocol  has  the  most  optimal 
throughput  value  and  measure  how  far  the  tradeoff  occurs  between  VXLAN  and  NVGRE  and 
also  with  native.  Based  on  Figure  3,  it  can  be  seen  that  there  is  difference  of  throughput  value 
between  VXLAN  and  NVGRE  performance.  The  difference  occurs  when  the  size  of  the  tested 
packet  size  is  256  -1514  bytes.  NVGRE  performance  has  a  greater  throughput  value  than 
VXLAN  performance  throughout  the  packet  size  testing  is  increased.  These  performance 
distance  is  due  the  different  characteristics  of  the  number  of  packet  header  use  for  each 
tunneling  platform,  the  NVGRE  has  4  byte  GRE  header,  and  a  16  byte  VXLAN  protocol 
consisting  of  8  bytes  of  VXLAN  headers  and  8  bytes  of  user  datagram  protocol  (UDP).  With  less 
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packet  headers,  its  proven  that  NVGRE  have  higher  throughput  performance  than  VXLAN.  For 
the  most  optimal  packet  size  performance,  both  of  tunneling  protocol  has  the  same  variable 
which  is  at  1514  bytes,  with  VXLAN  throughput  performance  of  908.55  Mbps  and  NVGRE  of 
916.60  Mbps.  It  caused  of  the  packet  size  can  impact  on  the  utility  of  bandwidth  usage,  so  when 
the  packet  size  is  1534  bytes,  each  protocol  gets  the  most  optimal  throughput. 

In  Figure  4  shows  that  the  result  has  the  different  delay  performance.  Each  tunneling 
protocol  has  a  tendency  to  increase  delay  value  when  packet  size  is  tested  larger.  Tunneling 
protocol  that  has  the  highest  delay  performance  is  VXLAN,  it  takes  on  packet  size  above  768 
bytes  with  delay  value  2.98  ms.  The  maximum  performance  of  delay  is  not  exceeding  3  ms,  so 
the  delay  performance  in  tunneling  protocol  on  virtualization  environment  is  very  reliable  to  use, 
because  the  delay  performance  accepted  with  ITU  G.1010  standard,  related  to  server  access, 
which  is  has  <2s  standard.  This  also  shows  that  the  performance  of  Open  vSwiitch  as  a  tunnel 
end  point  is  able  to  work  efficiently,  and  the  delay  performance  is  not  too  far  away  from  native 
performance.  Figure.  5  shows  Jitter  performance  from  VM-A  to  VM-B. 
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Figure.  4.  Delay  performances  from  VM-A  to  VM-B 


Figure.  5.  Jitter  performance  from  VM-A  to  VM-B 


There  are  two  sides  of  the  test  of  vCPU  usage  performance,  at  the  sender  (VM-A)  and 
recipient  side  (VM-B).  Testing  is  conducted  by  sending  traffic  from  VM-A  to  VM-B  by  using  a 
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different  variable  packet  size  as  shown  in  Figure  6.  For  the  test  results  at  the  sender  can  be 
seen  in  Figure  6.  From  the  images  shown  that  the  VXLAN  protocol  requires  the  largest  vCPU 
usage  compared  to  NVGRE  and  Native  performance.  The  performance  of  VXLAN  protocol  with 
NVGRE  has  the  same  trend  of  test  result,  that  is  the  bigger  packet  size  used  then  the  vCPU 
usage  performance  will  decrease.  This  is  due  to  the  greater  packet  size  used,  will  reduce  the 
ratio  of  the  number  of  packet  deliveries  per  second  to  the  bandwidth,  so  the  number  of  packets 
sent  per  second  will  decrease. 

With  the  decreasing  number  of  packets  per  second  transmitted,  it  will  be  directly  decline 
the  number  of  packets  that  must  be  encapsulated,  so  the  performance  of  vCPU  usage 
decreases.  At  the  test  result  on  VM-B  can  be  seen  in  Figure  7,  from  the  graphic  image  shows 
the  vCPU  usage  in  VM-B  as  a  whole  has  increased  performance  value  compared  to  VM-A,  this 
performance  improvement  shows  that  vCPU  usage  during  the  process  of  decapsulation  in 
receiver  (VM-B),  requires  more  vCPU  usage  than  the  sender's  encapsulation  process. 
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Figure  6.  vCPU  Usage  VM-A 


Figure  7.  vCPU  Usage  VM-B 


4.  Conclusion 

We  conclude  from  the  test  results  and  performance  analysis  of  tunneling  protocol  it 
shows  that  the  tunneling  protocol,  VXLAN  and  NVGRE  has  the  different  performances.  The  best 
tunneling  protocol  in  terms  of  bandwidth  usage  efficiency  is  the  NVGRE  tunneling  protocol,  with 
an  average  use  of  throughput  is  771,02  Mbps  while  VXLAN  has  753,62  Mbps.  Then  the  delay 
test  shows  that  the  tunneling  protocol  which  has  the  lowest  average  travel  time  is  the  NVGRE 
tunneling  protocol,  with  the  average  delay  value  of  2.24  ms,  while  the  VXLAN  is  2.29  ms.  The 
second,  delay  performance  of  this  protocol  is  in  accordance  with  ITU.TG.1010  related  on  server 
access  is  <2s.  In  jitter  performance  shows  that  the  tunneling  protocol  which  has  the  lowest 
variance  delay  rate  is  the  VXLAN  tunneling  protocol  with  jitter  value  is  0.348  ms  while  NVGRE 
performance  is  0.361  ms.  The  last  is  the  CPU  Usage  performance  shows  that  the  tunneling 
protocol  which  has  the  lowest  of  vCPU  usage  is  the  NVGRE  tunneling  protocol,  with  the 
average  CPU  Usage  value  on  the  sender  and  recipient  host  is  60.57%  and  the  VXLAN 
performance  is  79.84%. 

For  overall,  it  can  be  concluded  that  the  most  optimal  tunneling  protocol  performance  is 
NVGRE  because  its  superiority  from  3-of-4  parameters,  there  are  the  throughput,  vCPU  usage, 
and  delay  performance. 
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